Field of the Invention
The present invention relates to an image processing apparatus, an image capturing apparatus, a control method, and a recording medium, and particularly to a technique for generating an image that is focused at an arbitrary subject distance based on output data after shooting.
Description of the Related Art
In recent years, a technique has been proposed in which, in image capturing apparatuses such as digital cameras, an intensity distribution and traveling directions of light is recorded as output data during shooting so as to make it possible to generate an image, in which a focus position is positioned on an arbitrary image plane, based on the output data after recording.
Ren. Ng et al., “Light Field Photography with a Hand-held Plenoptic Camera”, Stanford University Computer Science Tech Report CTSR, 2005-02 discloses a method (Light Field Photography) in which light that is incident from various directions is recorded separately by causing light beams that pass through different divided pupil regions of an imaging lens to form images on the pixels (photoelectric conversion elements) of an image sensor via a microlens array. In the output data obtained in this way (light field data, which is hereinafter referred to as “LF data”), adjacent pixels record light beams that were incident from different directions.
By extracting light beams from the same direction from the pixels associated with each microlens, it is possible to generate an image captured from that direction based on the LF data. Also, by setting a focus position and adding up the output of the pixels that recorded the light beams that passed through one point in the focal plane which includes the focus position, it is possible to virtually generate (reconstruct) pixels for an image that is focused at a specified focus position (an image plane) after shooting.
Also, by using only pixels that received light beams that passed through a limited range of divided pupil regions in the LF data, it is possible to obtain an effect equivalent to the case where an aperture is applied in the imaging optical system. Specifically, by generating a reconstructed image using only pixels that correspond to divided pupil regions that correspond to the center of the exit pupil, it is also possible to generate an image that has a deep depth of field and is focused at a wide range of subject distances.
However, when a reconstructed image having a deep depth of field is generated as described above, there are cases where a subject other than the desired subject is also focused on. For example, a passerby or the like who is passing in front of or behind the person who is the desired subject can possibly be in-focus. In particular, a subject that is located closer to the image capturing apparatus during shooting than the desired subject is will have a greater presence in the reconstructed image.